Systems and methods for modifying modulated signals for transmission

ABSTRACT

Systems and methods are disclosed herein for modifying modulated signals for transmission. The system receives a modulated signal comprising a speech signal and a carrier wave and generates first and second spectral signals by converting the modulation signal and carrier wave from the time domain to the frequency domain respectively. The system then determines spectral bands for the first and second spectral signals. For each spectral band, the system calculates a weighted spectral band value based on a magnitude of the first spectral signal within the spectral band and generates a modified spectral signal by modifying the second spectral signal with the weighted spectral band value. The system then converts the modified spectral signal from the frequency domain to the time domain and transmits the converted modified spectral signal to a server.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.16/383,107, filed Apr. 12, 2019. The contents of which are herebyincorporated by reference herein in its entirety.

BACKGROUND

The present disclosure is directed to techniques for signal modulation,more particularly, modifying modulation signals to avoid featureextraction from digital speech data.

SUMMARY

Conventional signal processing approaches for modifying digital speechdata by modulation require substantial processing resources and/or timeexpenditure. These conventional signal processing techniques, such aswavelet techniques, may be used to modify digital speech data using aconvolution procedure requiring shifting signal phase, multiplication ofsignal portions, and integration of the signal portions. Each of thesestages requires significant processing resources to complete. Techniquesfor modulating digital speech data to avoid feature extraction (e.g.,anonymizing gender, pitch, and cadence) remain technically challenging,as conventional signal processing approaches cannot efficiently processthe digital speech data to prevent feature extraction.

Accordingly, techniques are disclosed herein for modifying modulatedsignals for transmission. The disclosed techniques herein discussreceiving a modulated signal including a speech signal and a carrierwave. First and second spectral signals are generated by converting thespeech signal and carrier wave from the time domain to the frequencydomain (e.g., using fast Fourier Transform). Spectral bands for thefirst and second spectral signals are determined. For each spectralband, a weighted spectral band value is calculated based on themagnitude of the first spectral signal within the spectral band. Thedisclosed techniques generate, for each spectral band, a modifiedspectral signal by modifying the second spectral signal with theweighted spectral band value. The modified spectral signal is convertedfrom the frequency domain to the time domain and then transmitted to aserver.

In some embodiments disclosed herein, the disclosed techniques executeweighting operations to the magnitudes for each of the frequencieswithin the spectral band. Specifically, the system determines aplurality of frequencies within the spectral band. Magnitudes are thencalculated for each of the plurality of frequencies within the spectralband. The system executes a weighting operation (e.g., a weightedaverage) of the magnitudes for each of the plurality of frequencieswithin the spectral band.

In some variants, the system determines spectral bands for the first andsecond spectral signals by determining spectral bands based onpredefined values. The system then assigns the determined spectral bandsto the first and second spectral signals such that both spectral signalshave the same spectral bands.

The techniques disclosed herein may be used as means to efficientlyanonymize speech signals. Recognizable features of speech signalsincluding gender, cadence, expression, inflections, and other audio cuesassociated with speech may be anonymized for further processing. Forexample, further processing may include speech-to-text extraction. Inthis scenario, the modified spectral signal sent to a speech-to-textextraction server results in the server processing an anonymized speechsignal with no features for extraction.

BRIEF DESCRIPTION OF THE DRAWINGS

The below and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows an illustrative diagram for modifying modulated signals fortransmission, in accordance with some embodiments of the disclosure;

FIG. 2 shows an illustrative diagram of a sawtooth wave in the timedomain and transformed into the frequency domain, in accordance withsome embodiments of the disclosure;

FIG. 3 shows an illustrative diagram of a modulated speech signal in thetime domain and transformed into the frequency domain, in accordancewith some embodiments of the disclosure;

FIG. 4 shows an illustrative diagram of a converted modified spectralsignal in the frequency domain and transformed into the time domain, inaccordance with some embodiments of the disclosure;

FIG. 5 shows an illustrative system diagram of the modification engine,speech service server, and multiple electronic devices, in accordancewith some embodiments of the disclosure;

FIG. 6 shows an illustrative block diagram of the modification engine,in accordance with some embodiments of the disclosure;

FIG. 7 is an illustrative flowchart of a process for selective audiosegment compression for accelerated playback of media assets, inaccordance with some embodiments of the disclosure; and

FIG. 8 is an illustrative flowchart of a process for executing aweighting operation to the magnitudes for each of the plurality offrequencies within the spectral band, in accordance with someembodiments of the disclosure.

DETAILED DESCRIPTION

FIG. 1 shows an illustrative diagram 100 for modifying modulated signalsfor transmission, in accordance with some embodiments of the disclosure.A modification engine receives a modulated signal 102 including both aspeech signal and a carrier wave. The speech signal may be any type ofdigital speech data including all elements of speech, including lexicaland non-lexical speech elements. The carrier wave may be any type ofcarrier wave including sinusoidal, sawtooth, or another type of carrierwave. In a preferred embodiment, a sawtooth carrier wave is used as thecarrier wave due to inherent harmonic richness, which is adaptable forcarrying a speech signal. The received modulated signal is derived fromthe signal processing modulation of the input speech signal and thecarrier wave. In some embodiments, the modulation is performed at themodification engine. In yet other embodiments, the modulation isperformed on a device prior to the modification engine receiving themodulated signal (e.g., on a device receiving speech input such as asmartphone/smartwatch, or a server that receives the speech input from afirst device and modulates the speech signal before transmitting themodulated signal to the modification engine).

The modification engine generates a first spectral signal by convertingthe modulation signal from the time domain to the frequency domain andthen determines the spectral bands for the first spectral signal. Asshown in FIG. 1, there are three filters F₁-F₃ applied to the modulatedsignal to define three spectral bands. Filters are selected based onspectral band selection. For each spectral band a weighted spectral bandvalue is calculated based on the magnitude of the first spectral signalwithin the spectral band. For example, the modification enginecalculates a weighted average of the magnitudes of the amplitudes of thefrequencies within the first spectral band, and subsequently for theother second and third spectral bands.

A modified spectral signal is generated by the modification engine, foreach spectral band, by modifying the second spectral signal (e.g., thecarrier wave 104) with the same filters used for the modulated signal toselect the same spectral bands from the carrier wave F₁-F₃. Both theseinputs are modified through modulation to generate respective modifiedspectral signals for each band. The modification engine performs asummation operation 106 for each of these modified spectral signals.

The modification engine then converts the summation of the modifiedspectral signals from the frequency domain to the time domain. This maybe performed using various conversion techniques such as inverse fastFourier Transform (IFFT) 108. The modification engine then transmits theconverted modified spectral signal to a server 110.

FIG. 2 shows an illustrative diagram 200 of a sawtooth wave in the timedomain and transformed into the frequency domain, in accordance withsome embodiments of the disclosure. Graph 210 illustrates a time domainrepresentation of a sawtooth carrier wave showing equal peaks ofamplitudes appearing periodically. Graph 220 illustrates a frequencydomain representation of the sawtooth carrier wave showing equal peaksof amplitudes appearing periodically across the measured spectral bandsB₁-B₃.

FIG. 3 shows an illustrative diagram 300 of a modulated speech signal inthe time domain and transformed into the frequency domain, in accordancewith some embodiments of the disclosure. Graph 310 illustrates a timedomain representation of a received modulated signal (including speechsignal). As illustrated, the amplitudes are irregular when analyzedperiodically, likely due to speech being irregular in tone and volume.Graph 320 illustrates a frequency domain representation of the modulatedsignal showing unequal peaks of amplitudes appearing periodically acrossthe measured spectral bands B₁-B₃.

FIG. 4 shows an illustrative diagram 400 of a converted modifiedspectral signal in the frequency domain and transformed into the timedomain, in accordance with some embodiments of the disclosure. Graph 410illustrates a frequency domain representation of the sawtooth wave(i.e., second spectral signal) modified with the weighted spectral bandvalue. Each spectral band may have a separate weighted spectral bandvalue. Thus, band B₁ is modified to increase the amplitude of the secondspectral signal, while band B₂ is modified to reduce the amplitude ofthe second spectral signal. Band B₃ is modified to increase theamplitude of the second spectral signal, however the magnitude ofincrease is less than the increase of band B₁. Graph 420 illustrates atime domain representation of the modified spectral signal (includingall converted bands B₁-B₃).

FIG. 5 shows an illustrative system diagram 500 of the modificationengine, speech service server, and multiple electronic devices, inaccordance with some embodiments of the disclosure. The modificationengine 504 may be of any hardware that provides for signal processingand transmit/receive functionality for signals. The modification enginemay be communicatively coupled to multiple electronic devices (e.g.,device 1 (506), device 2 (508), device 3 (510), device n (512)). Themodification engine may be communicatively coupled to a speech serviceserver 502. As illustrated within FIG. 5, the modification engine 504serves as a middleware between the electronic devices 506-512 and thespeech service server 502. A further detailed disclosure on themodification engine can be seen in FIG. 6 showing an illustrative blockdiagram of the modification engine, in accordance with some embodimentsof the disclosure.

In some embodiments, the modification engine may be implemented remotefrom the electronic devices 506-512 such as a cloud serverconfiguration. In yet other embodiments, the modification engine may beintegrated into electronic devices 506-512. In other variants, themodification engine may be integrated into the speech service server502. Any of the system modules (e.g., modification engine, speechservice server, electronic devices) may be any combination of shared ordisparate hardware pieces that are communicatively coupled.

The electronic devices (e.g., device 1 (506), device 2 (508), device 3(510), device n (512)) may be any device that has properties to transmitspeech signals. In other embodiments, the electronic devices may alsohave capabilities to transmit modulated signals including speech signalsand a carrier wave. The transmission may be analog or digital (includingdigital speech data). For example, the electronic device may be anyprocessor-based system, state machine, or retrofit network-connecteddevice. In various systems, devices can include, but are not limited to,network-connected devices (e.g., Internet-of-Things devices),smartphones, personal computers, smart appliances, consumer electronics,industrial equipment, security systems, digital twin systems, andsimilar systems or any combination of these systems.

The speech service server 502 may be any database, server, or computingdevice that contains memory for receiving signals containing speechsignals. The received signals may be unmodified or modified by amodification engine. In some variants, the speech service server may bea server providing services based on received speech signals (e.g.,Amazon Alexa server, Apple HomePod server, Microsoft Cortana server,Google Assistant server, virtual assistant servers, speech-to-textserver, and/or other voice command servers).

FIG. 6 shows an illustrative block diagram 600 of the modificationengine, in accordance with some embodiments of the disclosure. In someembodiments, the modification engine may be communicatively connected toa user interface. In some embodiments, the modification engine mayinclude processing circuitry, control circuitry, and storage (e.g., RAM,ROM, hard disk, removable disk, etc.). The modification engine mayinclude an input/output path 606. I/O path 606 may provide deviceinformation, or other data, over a local area network (LAN) or wide areanetwork (WAN), and/or other content and data to control circuitry 604,which includes processing circuitry 608 and storage 610. Controlcircuitry 604 may be used to send and receive commands, requests,signals (digital and analog), and other suitable data using I/O path606. I/O path 606 may connect control circuitry 604 (and specificallyprocessing circuitry 608) to one or more communications paths.

Control circuitry 604 may be based on any suitable processing circuitrysuch as processing circuitry 608. As referred to herein, processingcircuitry should be understood to mean circuitry based on one or moremicroprocessors, microcontrollers, digital signal processors,programmable logic devices, field-programmable gate arrays (FPGAs),application-specific integrated circuits (ASICs), etc., and may includea multi-core processor (e.g., dual-core, quad-core, hexa-core, or anysuitable number of cores) or supercomputer. In some embodiments,processing circuitry may be distributed across multiple separateprocessors or processing units, for example, multiple of the same typeof processing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). In some embodiments, control circuitry 604 executesinstructions for a modification engine stored in memory (e.g., storage610). In some embodiments, the processing circuitry provides for digitalsignal processing (DSP) processors by integrating specific hardware(e.g., Texas Instruments C6000 series DSPs, Freescale DSPs, AnalogDevices SHARC-based DSPs, and Huarui-2 processors by Nanjing ResearchInstitute of Electronics Technology). The DSP processors may bededicated integrated circuit chips.

Memory may be an electronic storage device provided as storage 610,which is part of control circuitry 604. As referred to herein, thephrase “electronic storage device” or “storage device” should beunderstood to mean any device for storing electronic data, computersoftware, or firmware, such as random-access memory, read-only memory,hard drives, solid state devices, quantum storage devices, or any othersuitable fixed or removable storage devices, and/or any combination ofthe same. Nonvolatile memory may also be used (e.g., to launch a boot-uproutine and other instructions). In some embodiments, the memory for DSPmay include Harvard architecture or Modified von Neumann architecture.

The modification engine 602 may be coupled to a communications network.The communication network may be one or more networks including theInternet, a mobile phone network, mobile voice or data network (e.g., a5G, 4G or LTE network), mesh network, peer-2-peer network, cablenetwork, or other types of communications network or combinations ofcommunications networks. Paths may separately or together include one ormore communications paths, such as a satellite path, a fiber-optic path,a cable path, a path that supports Internet communications, free-spaceconnections (e.g., for broadcast or other wireless signals), or anyother suitable wired or wireless communications path or combination ofsuch paths.

FIG. 7 is an illustrative flowchart of a process for selective audiosegment compression for accelerated playback of media assets, inaccordance with some embodiments of the disclosure. Process 700, and anyof the following processes, may be executed by control circuitry 604(e.g., in a manner instructed to control circuitry 604 by themodification engine). Control circuitry 604 may be part of modificationengine 602, or of a remote server separated from the modification engineby way of a communication network, or distributed over a combination ofboth.

At 702, the modification engine 602, by control circuitry 604, receivesa modulated signal comprising a speech signal and a carrier wave. Insome embodiments, the modification engine receives the modulated signalthrough the I/O path 606 which is coupled to an electronic device506-512. In some embodiments, the modification engine, by controlcircuitry 604, selects a carrier wave. The carrier wave may be asawtooth wave, sinusoidal wave, or any other type of wave. In yet otherembodiments, the modification engine is assigned a carrier wave throughthe I/O path 606.

At 704, the modification engine 602, by control circuitry 604, generatesa first spectral signal by converting the modulation signal from thetime domain to the frequency domain. In some embodiments, themodification engine uses processing circuity 608 to convert themodulation signal from the time domain to the frequency domain (e.g.,applying techniques such as fast Fourier Transform).

At 706, the modification engine 602, by control circuitry 604, generatesa second spectral signal by converting the carrier wave from the timedomain to the frequency domain. In some embodiments, the modificationengine uses processing circuity 608 to convert the modulation signalfrom the time domain to the frequency domain (e.g., applying techniquessuch as Fourier Transform).

At 708, the modification engine 602, by control circuitry 604,determines spectral bands for the first spectral signal and the secondspectral signal. In some embodiments, the modification engine utilizesprocessing circuity 608 to determine the spectral bands. In someembodiments, the modification engine retrieves predefined values from adatabase through the I/O path 606. The database may be integrated in themodification engine, a third-party server, integrated into devices506-512, or any other data structure that stores predefined values fordetermining spectral bands. The modification engine 602, by controlcircuitry 604, determines spectral bands for the first spectral signaland the second spectral signal based on predefined values. Themodification engine, by control circuitry 604, then assigns thedetermined spectral bands to the first spectral signal and the secondspectral signal. The assignment of the spectral bands is stored instorage 610. In some embodiments, when determining spectral bands forthe first spectral signal and the second spectral signal, themodification engine 602, by control circuitry 604, selects one or morefilters to create the determined spectral bands. In some embodiments,the selection of the filters is performed at least in part by processingcircuitry 608. In some embodiments, the selection of the filters isprovided to the modification engine through the I/O path 606. Themodification engine 602, by control circuitry 604, modifies the firstspectral signal and the second spectral signal based on the selected oneor more filters to create the determined spectral bands. In someembodiments, the modification of the first and second spectral signalsis performed at least in part by processing circuitry 608. In someembodiments, the one or more filters comprise at least one of a low passfilter, band pass filter, and high pass filter.

At 710, the modification engine 602, by control circuitry 604, for eachspectral band, calculates a weighted spectral band value based on amagnitude of the first spectral signal within the spectral band for eachspectral band. In some embodiments, the calculation of the weightedspectral band value is performed at least in part by the processingcircuitry 608. A further detailed disclosure on calculation of aweighted spectral band value based on a magnitude of the first spectralsignal within the spectral band for each spectral band can be seen inFIG. 8 showing an illustrative flowchart of a process for executing aweighting operation to the magnitudes for each of the plurality offrequencies within the spectral band, in accordance with someembodiments of the disclosure.

At 712, the modification engine 602, by control circuitry 604, for eachband, generates a modified spectral signal by modifying the secondspectral signal with the weighted spectral band value. In someembodiments, the modifying the second spectral signal with the weightedspectral band value is performed by the processing circuitry 608. Insome embodiments, the modification engine 602, by control circuitry 604,modifies the second spectral signal by amplitude modulation, frequencymodulation, or phase modulation.

At 714, the modification engine 602, by control circuitry 604, convertsthe modified spectral signal from the frequency domain to the timedomain. In some embodiments, the conversion of the modified spectralsignal from the frequency domain to the time domain performed by theprocessing circuitry 608. In some embodiments, the conversion techniqueused is inverse fast Fourier Transform.

At 716, the modification engine 602, by control circuitry 604, transmitsthe converted modified spectral signal to a server. In some embodiments,the modification server utilizes the I/O path 606 to transmit theconverted modified spectral signal to the server (e.g., speech serviceserver 502).

FIG. 8 is an illustrative flowchart of a process 800 for executing aweighting operation to the magnitudes for each of the plurality offrequencies within the spectral band, in accordance with someembodiments of the disclosure. At 802, the modification engine 602, bycontrol circuitry 604, determines a plurality of frequencies within thespectral band. In some embodiments, the modification engine determines aplurality of frequencies within the spectral band using processingcircuitry 608.

At 804, the modification engine 602, by control circuitry 604,calculates the magnitudes for each of the plurality of frequencieswithin the spectral band. In some embodiments, the modification enginecalculates the magnitudes for each of the plurality of frequencieswithin the spectral band using processing circuitry 608. For example,the processing circuitry 608 may have specific DSP processors toefficiently calculating the magnitudes for each of the plurality offrequencies.

At 806, the modification engine 602, by control circuitry 604, executesa weighting operation to the magnitudes for each of the plurality offrequencies within the spectral band. In some embodiments, themodification engine executes the weighting operation using processingcircuitry 608. For example, the processing circuitry 608 may havespecific DSP processors to efficiently perform the weighting operations.In some embodiments, the weighting operations include at least one of aweighted average, mean calculation, standard deviation, mediandetermination, mode determination, arithmetic operations, andstatistical operations.

It is contemplated that the steps or descriptions of FIGS. 7-8 may beused with any other embodiment of this disclosure. In addition, thesteps and descriptions described in relation to FIGS. 7-8 may be done inalternative orders or in parallel to further the purposes of thisdisclosure. For example, each of these steps may be performed in anyorder or in parallel or substantially simultaneously to reduce lag orincrease the speed of the system or method. Any of these steps may alsobe skipped or omitted from the process. Furthermore, it should be notedthat any of the devices or equipment discussed in relation to FIGS. 5-6could be used to perform one or more of the steps in FIGS. 7-8.

The processes discussed above are intended to be illustrative and notlimiting. One skilled in the art would appreciate that the steps of theprocesses discussed herein may be omitted, modified, combined, and/orrearranged, and any additional steps may be performed without departingfrom the scope of the invention. More generally, the above disclosure ismeant to be exemplary and not limiting. Only the claims that follow aremeant to set bounds as to what the present invention includes.Furthermore, it should be noted that the features and limitationsdescribed in any one embodiment may be applied to any other embodimentherein, and flowcharts or examples relating to one embodiment may becombined with any other embodiment in a suitable manner, done indifferent orders, or done in parallel. In addition, the systems andmethods described herein may be performed in real time. It should alsobe noted that the systems and/or methods described above may be appliedto, or used in accordance with, other systems and/or methods.

What is claimed is:
 1. A method for modifying a speech signal fortransmission comprising: receiving the speech signal; converting thespeech signal from the time domain to the frequency domain; determiningspectral bands for the speech signal and a spectral signal; for eachspectral band, generating a modified spectral signal by modifying thespectral signal based on a magnitude of the speech signal; convertingthe modified spectral signal from the frequency domain to the timedomain; and causing to transmit the converted modified spectral signalto a server.
 2. The method of claim 1, wherein modifying the spectralsignal based on a magnitude of the speech signal comprises: calculatinga weighted spectral band value based on the magnitude of the speechsignal; determining a plurality of frequencies within each spectralband; calculating the magnitudes for each of the plurality offrequencies within each spectral band; and executing a weightingoperation to the magnitudes for each of the plurality of frequencieswithin each spectral band.
 3. The method of claim 2, wherein theweighting operation comprises at least one of a weighted average, meancalculation, standard deviation, median determination, modedetermination, arithmetic operations, and statistical operations.
 4. Themethod of claim 1, wherein the modification of the spectral signalcomprises at least one of amplitude modulation, frequency modulation,and phase modulation.
 5. The method of claim 1, wherein modification ofthe spectral signal comprises: determining the spectral bands for thespeech signal based on predefined values; and assigning the determinedspectral bands to the speech signal.
 6. The method of claim 5, furthercomprising: selecting one or more filters to create the determinedspectral bands; and modifying the speech signal based on the selectedone or more filters to create the determined spectral bands.
 7. Themethod of claim 6, wherein the one or more filters comprise at least oneof a low pass filter, band pass filter, and high pass filter.
 8. Themethod of claim 1, wherein the speech signal is generated by convertinga modulation signal from the time domain to the frequency domaincomprises a fast Fourier Transform.
 9. The method of claim 1, whereinconverting the modified spectral signal from the frequency domain to thetime domain comprises an inverse fast Fourier Transform.
 10. The methodof claim 1, wherein generating a modified spectral signal by modifyingthe spectral signal based on a magnitude of the speech signal comprisesmodulating the spectral signal with a sawtooth wave.
 11. A system formodifying a speech signal for transmission comprising: control circuitryconfigured to: receive the speech signal; convert the speech signal fromthe time domain to the frequency domain; determine spectral bands forthe speech signal and a spectral signal; for each spectral band,generating a modified spectral signal by modifying the spectral signalbased on a magnitude of the speech signal; convert the modified spectralsignal from the frequency domain to the time domain; and cause totransmit the converted modified spectral signal to a server.
 12. Thesystem of claim 11, wherein the control circuitry is configured, whenmodifying the spectral signal based on a magnitude of the speech signal,to: calculate a weighted spectral band value based on the magnitude ofthe speech signal; determine a plurality of frequencies within eachspectral band; calculate the magnitudes for each of the plurality offrequencies within each spectral band; and execute a weighting operationto the magnitudes for each of the plurality of frequencies within eachspectral band.
 13. The system of claim 12, wherein the weightingoperation comprises at least one of a weighted average, meancalculation, standard deviation, median determination, modedetermination, arithmetic operations, and statistical operations. 14.The system of claim 11, wherein the modification of the spectral signalcomprises at least one of amplitude modulation, frequency modulation,and phase modulation.
 15. The system of claim 11, wherein the controlcircuitry is configured, when modifying the spectral signal, to:determine the spectral bands for the speech signal based on predefinedvalues; and assign the determined spectral bands to the speech signal.16. The system of claim 15, wherein the control circuitry is furtherconfigured to: select one or more filters to create the determinedspectral bands; and modifying the speech signal based on the selectedone or more filters to create the determined spectral bands.
 17. Thesystem of claim 16, wherein the one or more filters comprise at leastone of a low pass filter, band pass filter, and high pass filter. 18.The system of claim 11, wherein the speech signal is generated byconverting a modulation signal from the time domain to the frequencydomain comprises a fast Fourier Transform.
 19. The system of claim 11,wherein converting the modified spectral signal from the frequencydomain to the time domain comprises an inverse fast Fourier Transform.20. The system of claim 11, wherein generating a modified spectralsignal by modifying the spectral signal based on a magnitude of thespeech signal comprises modulating the spectral signal with a sawtoothwave.